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(54) Control system 

(57) A control system primarily for patients having limited mobility and limb movement, comprises a computer controlled 
display of menu sequences each showing a sequence of utility items and selectable by a cursor. Control of the cursor and 
activalion of the utility item so selected Is effected by the patient by means of a pressure sensor, either mouth or finger 
operated or both, in conjunction with an eye direction sensor, the two sharing the selection and activation processes at the 
will of the patient The patient's sensors are United to the computer by an infra-red link so that there is no fear of 
electrocution and also to avoid the inconvenience and danger of trailing cables. 
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At least one drawing originally filed was infomial and the print reproduced here is taken from a later filed formal copy. 
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Fig. 3. 
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Fig. 5 (a) 
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Control System 



This invention relates to a control system providing an 
interface between a human being and a range of machine and electronic 
devices, that is, a form of cybernetic system, and particularly to such a 
system for use by a human operator whose activities are restricted either 
by personal incapacity or by the demands of the environment in which he is 
operating. Typical examples are paraplegic or quadriplegic people and 
persons having restricted mobility due to limited confinement who require 
the use of an or most of their limbs (including head-movement) for other 
purposes, eg operators of bathyspheres, mini -submarines, space capsules, 
aircraft cockpits and simulators related to these systems. 

Previous arrangements for use in circumstances of the 
above kind have included electrode devices for attachment to the 
operator's head to sense charge variations on the eyeball. The resulting 
electrical signals may then be coded, by their duration, rate of change 
etc. to indicate requirements. While such arrangements may be 
satisfactonr in specific circumstances th^ are of limited application. 
Dependence on the eyes alone for this control function can become irksome, 
especially when the operator is an invalid. A further disadvantage of 
known systems of this kind is the need for trailing connections between 
the operator's 'head set' and the computer. 



Such connections can constitute an actual nuisance and, perhaps more 
important, a psychological stress, in that there is a direct connection 
between the operator's head and a computer system which is, or appears to 
be, fed from 'the mains'. 

An object of the present invention is therefore to 
provide a control system of the above type which significantly improves 
the operator's control capability while still permitting operation 
according to selected Individual modes. 

According to the present Invention, a control system for 
use by a human operator comprises coiq)uter means adapted to control a 
range of utility items, displ^ means controlled by the computer means and 
adapted to displ^ at least one menu of the utility items, pressure sensor 
means adapted to be operated by the operator, ^e-di recti on sensor means 
adapted to be fitted to an operator's head the pressure sensor means and 
the eye-direction sensor means being coupled to the computer means by an 
electro-magnetic link to control selection and/or activation of the 
utility items. 

The electromagnetic link is preferably an infra-red link. 

The display means mdiy be adapted to displeiy two or more 
stages of menu, a menu of the second or a subsequent stage constituting a 
the utility item of a menu of the preceding stage. 

The pressure sensor may be an air pressure sensor 
responsive to pressures above and below atmospheric produced by blowing 
and sucking. A cursor may be movable over displayed menu Items by the 
sucking or the blowing process and activation of a selected menu. Item 
effected by the complementary process. 

The eye-direction sensor means and the pressure sensor 
means may control, one, the selection of menu Items, and the other, the 
activation of selected menu Items. 

The computer means may be adapted to scan menu items 
automatically until an item is selected by operation of the pressure 
sensor means. 



The control system may be adapted to operate in an eye 
control mode, the ^e-direct1on sensor means controlling the selection of 
menu items, and the system including means responsive to the dwell time of 
the eye direction to activate any menu item subjected to the same eye 
direction for a predetermined period. 

The control system may further include means responsive 
to the closure of one but not two eyes to switch between operational 
modes, and particularly from a standby to an active mode. 

A control system for use by a human operator Whose 
mobility is restricted by environment or personal incapacity will now be 
described, by way of example, with reference to the accompanying drawings, 
of which: 

Figure 1 is a block diagram of a preferred arrangement 
providing a choice between, or combination of eye direction sensor and 
pressure sensor; 

Figure 2 is a diagram of an abbreviated version of the 
arrangement of Figure 1 showing a pressure sensor and pressure transducer 
In some detail; 

Figure 3 is a control mode menu to be displayed to the 
Figure 4 is a main menu following autoraatlcally from 
Figure 5(a) is a particular sub-menu of a typewriter 
Figure 5(b) is a sub-menu expansion of part of the menu 
Figure 6 Is a sub-menu resulting from selection of radio 



operator; 
Figure 3; 
mode; 

of Figure 5(a); 
equipment! 



and Figures 7 and 8 show, respectively, transmitting and 

receiving units of an I.R. link. 

Referring to Figure 1 the system comprises an air 
pressure sensor unit 1. shown in detail in Figure 2. coupled to a computer 
3 by way of an infra-red link 5. The computer controls a display 7 and 



( 
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also a number of utility devices 9 which may include radio, TV, recorder, 
fan, lights, telephone, motor/servo units, robots, wheelchairs and many ' 
more. 

The computer is progranroed to associate the utility 
devices 9 with items on a menu displayed by the display system 7. A 
cursor on the display is moved to select a particular item which is 
associated either with further displ^s in sub-menus or with a particular 
utility device 9 directly. An activation signal me(y be provided by the 
pressure sensor unit 1 to the computer 3 to activate the selected Item, 1e 
the selected utility device. 

An eye-direction sensor system 11 is incorporated to rive 
additional control and flexibility to the overall system. This ^e-sensor 
system comprises a pair of electrodes 13 which are fitted to the 
operator's head In close contact with his temples. The charge 
distribution around a person's eyes Is affected by the 
■d1rert1on-of-look*. Suitably placed electrodes (13) can therefore be 
employed to sense this charge disturbance and hence the direction-of-look. 
The potential difference arising can be used to control the cursor of the 
display system 7 by wc(y of the computer 3 to perform the selection and, as 
will be seen, the activation, of menu items. 

In an extreme case where the pressure sensor cannot be 
used, the activation of a selected menu Item can be achieved by responding 
to the dwell time of th? eyes on the selected item. A sufficiently long 
dwell time. In excess of a predetermined threshold, produces an activation 
signal* 

More often however, the eye sensor would be used for only 
one of the two functions, selection and activation and most conveniently 
that one would be the selection function. An advantage of the system Is 
however the flexibility prov Ided by the ability to choose either eiye 
sensor or pressure sensor or a combination, for selection and activation. 

Referring now to Figure 2, the pressure sensor unit 1 of 
Figure 1 comprises essentially a suck/blow mouthpiece 15 and a pressure 
transducer 17 which converts the pressure signals to electrical signals. 



A manually operable valve 19 enables the mouthpiece to be isolated from 

the transducer 17. 

A further pressure sensor 21, which may be one of a 
series connected to a common closed tube 23. permits a pressure signal to 
be sent to the pressure transducer by means of. say. finger contact on an 
elastic surface. Operation of such sensors does of course require the 
suck/blow mouthpiece 15 to be isolated by the valve 19. The suck/blow 
mouthpiece and the tactile sensor 21 are shown associated vrith parallel 
branch tubes. They could, of course, be fitted to the same tube. 

The pressure transducer 17 responds to both positive and 
negative pressure, that is. pressure above and below atmospheric. 
Positive pressure is sensed by a normally open contact 25 coupled to a 
diaphragm, not shown, and negative pressure is sensed by a normally open 
contact 27 coupled to a diaphragm also not shown. 

The two contacts provide three output lines: a comnon 
line 29 connected to both 'wiper' contacts; and two lines 31 and 33 
connected to the respective other contacts. These three lines are 
connected to control an infra red transmitter 36 linked tyo a remote infra 
red receiver 37 as in Figure 1. to control the selection and activation of 
the displayed menu items by the computer. As shown, selection is achieved 
by the positive pressure switch 25. and activation by the negative 
pressure switch 27. ie 'blow' for selection and 'suck' for activation. At 
the wish of the operator, this arrangement can be reversed by a reversing 
switch 35 which interchanges the signals on the lines 31 and 33. 

The use of an I.R. link has the advantage of making the 
sensor systems self-contained and local to the operator. There Is no wire 
connection to any equipment which is fed from 'the mains'. The I.R. 
receiver 37. the computer 3. display 7 and Interface with the utility 
devices 9 are all relatively remote from the operator and he can feel 
secure from any high voltages that th^ might exhibit. 

There Is also the advantage that no trailing wires, 
cables etc. extend from the operator to the control equipment to 
constitute an accident hazard. 



As described above, the selection of a menu Item was 
achieved by steering the cursor horizontally either by picking off a 
voltage from the eye direction sensor or by control via the positive 
pressure (blow) signal. An alternative method that is available in 
accordance with the invention is for the computer to control an automatic 
scan. Since no vertical control is required (manually) the scan can be 
two dimensional and cover a matrix or arrs^y of menu items. The operator 
then merely has to activate the system when the required item is Mit 
upon\ The scan may be of the same form as the conventional TV scan but 
with only 6 to 10 (say) lines per frame. The scan is then cyclic and may 
be operable at two speeds. 

Figure 3 shows a suitable control mode menu for setting 
up the mode of operation according to the operator's capabilities or 
preference. The menu comprises the following control modes: 

(1) Eyescan. 

This requires the eye-direction sensor, which is 
calibrated to the individual operator, and a mechanism such as the 
pressure sensor described above for activation of the menu item selected 
by the eye-sensor. Other mechanisms, such as a sound activation unit or a 
joystick, may be incorporated. The essential operation is however, 
selection by eye direction and activation by other mechanism. 

(2) Autoscan fast and slow. 

This has been described above, the menu items being 
scanned automatically and selection and activation effected by operation 
of the pressure mechanism at the appropriate time. There are two scan 
rates available. 

(3) 'Vacuscan*. 

Here the selection and activation is performed manually, 
both by the same mechanism, eg the blow/suck pressure device m^ be used 
to select by blowing and to activate by sucking. In the selection process 
each 'blow* causes the cursor to move along the one-dimensional arr^ of 
menu items in predetermined steps. 



(4) "Eye pause*. 

This iBOde involves only the ^e-direction sensor, 
selection is performed as in (1) above but activation is also achieved by 
means of the eye-direction sensor. Thus, on selecting a particular menu 

looKing directly at the item, the operator then holds the posU ion 
for a predetermined period, eg three seconds. A timer is then triggered 

and the selected item activated. 

The eye-direction sensor means is sensitive both to the 
•direction-of-look' and to the open/closed state of the eye. the detected 
voltage levels corresponding distinctively to these conditions.A control 
signal can thus be provided by blinking the eyes. However, to avoid 
ambiguity arising from involuntary blinking, an exclusive-OR function is 
included to detect only single-eye blinking, ie a 'wink'. A mi n mum wink 
duration is also Imposed to ensure that the movement is del berate. 

The control mode may be arranged to default to a 
particular one of the modes whon in Figure 3 either to suit the particular 
operator or. in general, the least demanding mode, ie the Eye-pause mode. 
The operator, initially presented with the Figure 3 menu, may then keep 
the eye-pause mode or. using that mode select one of the others. 

Each of the control modes when selected and activated is 
followed by a menu such as that of Figure 4. The three menu Items shown 
are TYP for 'typewriter mode'; BYW for 'standby .»de'; and DEV for 'device 
mods * 

Activation of the typewriter mode causes a menu such as 
shown in Figurte 5(a) to be displayed, this being a t^''*-^^^*;^"^ 

The items consist of the alphabet, various editing items, 
and an outlet to the next menu, 'page 2'. The 30 items are arranged in 6 
columns and the cursor 39 can be controlled to move horizontally by 
blowing on the pressure sensor mouthpiece. A particular column is 
selected by positioning the cursor and. assuming operation in the eye-scan 
mode, this selection activated by sucking on the mouthpiece. The selected 
column is then displayed in a horizontal row In the menu. A further 
cursor selection and activation is performed so selecting a particular 
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Item from the row* A sequence of letters can thus be called up to form a 
piece of text which occupies the "text workspace" area« This my then be 
printed out In a hard copy by selection and activation of the "print" Item 
In column 6 of the original menu. 

Figure 5(b) shows the sub menu for a selection of the 
sixth column of Figure 5(a) and shows the cursor positioned under "page 
2"» This selection is activated as before by sucking on the mouthpiece to 
give the menu denoted PA2 (page 2) which m^ be related to the text 
writing function or may be an exit to a previous or other menu. 

The operator m^y require a rest In the middle of some 
(particularly typewriting) operation. The BYE (standby) mode of Figure 4 
provides this function by effectively storing the current state of an 
operation until such time as the operator wishes to come back to it. When 
In the standby mode and also in the eye-^pause control mode, exit from the 
standby mode is effected by a wink, which returns the operation to the 
state immediatgely prior to the rest period. 

The third operational mode available from the menu of 
Figure 4 is the device control mode (DEV), Selection of this mode 
produces a menu comprising a list of devices that can be remotely 
controlled by the computer, eg radio, telephone, door locks etc. These 
items are selected and activated as required. One further menu available 
as an adjunct to, say, the radio facility, is shown In Figure 6. This 
comprises a linear scale of frequency which can be selected for tuning a 
radio In either an analogue or digital manner. The selected frequency Is 
displayed. 

Each menu available has an exit, which Is either 
automatic, on a time-out, or Is selectable as one menu Item to enable the 
operator to revert to the main menu, to proceed to an associated menu or 
perhaps to close down. 

Figures 7 and 8 show respectively the transmitting and 
receiving units of the infra red link. 



The transmitter, local to the operator, is coupled in 
this instance to a joystick device 41 which produces left/right and 
up/down output signals according to the direction of hand pressure 
applied. The resulting signals are encoded by an integrated circuit unit 
43 and arranged to pulse a pair of I.R. light-eraitting-diodes 45 
accordingly. 

Figure 8 shows the corresponding circuitry of the 
receiver unit local to the computer and fed by a photo-sensitive diode 47. 

The use of ajoystick in this embodiment does simplify the 
selection of (particularly textual) items from a rectangular array, 
removing one stage of selection. It may well, however, be beyond the 
capabilities of certain handicapped operators. 

With regard to the application of the described system to 
hospital patients, whatever the injuries, disease, disabilities, 
confinement conditions or requirements for the use of limbs, in by far the 
largest number of cases, movement of the direction of look of the eyes is 
the last ability to be unavailable. Hence the system utilises the 
measurement of direction of look of the ?yes in its main mode. However, 
it has been found that the effort of concentration needed if this is the 
only property that is used is too great for patients to use for usefully 
long and frequent periods and it has been found that this is due to the 
relatively short period of steady visual concentration that will be 
tolerated ty a patient in the condition postulated. 

Hence the system incorporates the 'visual -only' mode as a 
back-up providing conmuni cation and control for the rare cases where the 
^es are the only available movement that the patient can control. There 
are also rare cases, eg eye damaged patients, in which eye movement is not 
adequately available but some other movement is. A minimum movement, 
minimum pressure sensor as described above is used to control an auto-scan 
of the display and select the menu item. In this w^y. the rare extreme 
cases are catered for, even if the best in conrnini cation and control that 
can be obtained for a patient in that condition is limited. 
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The great majority of cases where eye control Is In use 
will utilise the main mode above which enables, without significant 
patient strain, almost continuous and fully effective operation In both 
communication and control as often as the patient or medical authority 
require. 

This mode uses both the eye sensor and the pressure 
sensor In such a ^ay that the scan of the dlsplaiy Is carried out by the 
eye direction-of-look measurement and the activation of selected Items 
from the scan Is carried out by the minimum-movement-minimum-pressure 
sensor. In this way, the eyes are given continual rests from 
concentration when not scanning the d1sp1i(y prior to selection or after 
selection. It is found a patient's concentration fades If eye control is 
attempted continuously after too short a period In most cases. Several 
orders better usage can be obtained If rests and alternative control 
procedures are provided both to eye control and to control by the 
force/pressure switch, and by utilising both at once and also providing 
autoscan. The main mode Is based on this mix of techniques. 

In all modes of the system, progressive menu driving. 
Involving main menu, sub-menus and sub-sub-menus etc, each with clear 
"repeat to previous level' Indication and simple clarity throughout is an 
Inherent requirement and provision in the preferred system. If not 
provided, the complications to both patient and medical authority are 
found to militate against actual use of the equipment. Wide spacing is 
used between display Items. These combinations allow elimination of the 
need to monitor head movement In this proposed system and allows the use 
of smaller, standard sensor designs, eg ECG sensors, which the patient 
will tolerate much more readily. In large-sensor designs, it has been 
found the patient tolerates them not at all. 
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CLAIMS 



A control system for use by a human operator and 
comprising conputer means adapted to control a range of utility items, 
display means controlled by said computer means and adapted to display at 
least one menu of said utility items, pressure sensor means adapted to be 
operated by the operator, ?ye-direction sensor means adapted to be fitted 
to an operator's head said pressure sensor means and said eye-direction 
sensor means being coupled to said computer means by an electro-magnetic 
link to control selection and/or activation of the utility items. 

2, A control system according to Claim 1, wherein said 
electromagnetic link is an infra-red link. 

3. A control system according to Claim 1 or Claim 2, wherein 
slid display means is adapted to display two or more stages of menu, a 
menu of the second or a subsequent stage constituting a said utility item 
of a menu of the preceding stage. 

4^ A control system according to any preceding claim, 

wherein said pressure sensor is an air pressure sensor responsive to 
pressures above and below atmospheric produced by blowing and sucking. 

5, A control system according to Claim 4. wherein a cursor 

il movable over displayed menu items by the sucking or the blowing process 
and activation of a selected menu item is effected by the complementary 
process. 

A control system according to any of Claims 1 to 4, 
wherein said ^e-d1rect1on sensor means and said pressure sensor means 
control, one, the selection of menu items, and the other, the activation 
of selected menu items. 
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7. A control system according to ar^y of Claims 1 to 4, 
wherein said computer means Is adapted to scan menu Iten^ automatically 
until an Item Is selected by operation of said pressure sensor means. 

8. A control system according to Claim 1 adapted to operate 
in an eiye control mode* said eiye-d1 recti on sensor means controlling the 
selection of menu items, and the system Including means responsive to the 
dwell time of the eye direction to activate ar\y menu Item subjected to the 
same etye direction for a predetermined period. 

9. A control system according to Claim 8, further including 
means responsive to the closure of one but not two eyes to switch between 
operational modes. 

10. A control system according to Claim 9» wherein closure of 
one but not two ^es effects a switch from a standby to an active mode. 

11. A control system substantially as hereinbefore described 
with reference to a^y of the embodiments described herein. 
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